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ABSTRACT 

This report describes a procedure for developing 
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simultaneous audiovisual instruction^ the motoric elements created by 
practice do not become an integrated part of the concept of building 
the object formed from the audiovisual information. Instead^ motoric 
elements remain "outside" the concept. Motoric elements can be 
integrated into the visual and linguistic concept only with more 
extensive practice^ or when practice and audiovisual instruction are 
presented sequentially and not simultaneously. One figure and seven 
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ABSTRACT 



A goal in this researcii is to learn how to use high-technology computer 
systems effectively in education and training. This paper reports a 
procedure for developing multimedia instructions that are "optimized" 
according to certain criteria. It describes the design of interactive 
videodisc-based assembly instructions using IBM's InfoWindow system. 
Research comparing interactive videodisc-based assembly instructions 
and passive videotape instructions is presented. Performance in building 
from memory an 80-piece object (made from the Fischer-Technik 
assembly kit) is assessed for six different groups. The main comparison 
is between groups which have interactive instructions and are allowed to 
build during training, and groups which have the same instructions but are 
noi allowed to build during training. The "build" groups never perform 
better from memory than the "no build" groups (on structure or efficiency) 
and sometimes perform significantly worse! An account is offered using 
a framework for multimedia concepts in memory: When practice is mixed 
with simultaneous audiovisual instruction, the motoric elements created 
via practice do not become an integrated part of the concept of building 
the object formed from the audiovisual information. Instead, motoric 
elements remain "outside" the concept. Motoric elements can be integrated 
Into the (visual and linguistic) concept only with more extensive practice, 
or when practice and audiovisual instruction are presented sequentially 
and not simultaneously. 
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I. introduction 



The main goal of our current research is the implementation of an 
interactive multimedia tutoring system for assembly, repair, and understanding 
of real physical objects (Baggett, Ehrenfeucht, & Hanna, 1987). As a preliminary 
to the so-called "intelligent" implementation, we worked first with interactive 
videodisc-based instructions which try to teach people to build an 80-piece 
object, called a lift, made from the Fischer-Technik assembly kit. The lift is 
shown in Figure 1 . The instructions were not meant to be "intelligent" in the 
sense of Sleeman and Brown (1982). For example, there was no diagnosis of 
errors, nor were there different levels of instructions for different subjects. 
But, as we will show, the conceptual units (the breakdown of the lift into parts 
and subparts) and the names for the units were derived from "typical" subjects 
and are the'-efore "natural." The system and software we used were provided by 
IBM. In this early phase of the work, we wanted to give the equipment and 
software a test run, and to learn how to use multimedia computer-aided 
instruction efficiently in procedural tasks. 



Insert Fig. 1 about here. 



This article compares the performance of six groups of subjects in building 
from memory. One group watched a 27-min videotape showing step-by-step 
construction of the lift, and then built it from memory. The other five groups 
watched interactive videodisc-based instructions showing the same images as 

ERIC 7 



page 2 

the videotape. These groups could replay segments and pause the image, using a 
touch screen, as will be explained below. Three of the five interactive groups 
built the lift once on-line, as they watched, and then a second time from memory. 
The remaining two interactive groups were not allowed to build on-line, but 
could only replay and pause; they built only once, after training, from memory. In 
all groups our interest was in memory performance. 

We first give an overview of the computer equipment and the stimulus 
materials, including the design of the passive and interactive presentations. 
Predictions about performance from memory are made from our theoretical 
framework. We then report the very surprising results and the practical and 
theoretical conclusions. 

II. Computer Equipment and Stimulus Materials 

A. Equipment 

The equipment p-ovided by IBM is their InfoWindow system. Ours is 
XT-based, with a special monitor and videodisc. Input can be via keyboard and 
touch screen. Output is moving and still video with text and/or color graphics 
overlay. There are three sources of speech: two from the two videodisc 
soundtracks, and one from a limited speech synthesizer. IBM also provided 
canned software, called Composer/Conductor, for designing and delivering 
videodisc-based presentations. Composer has a spread-sheet design and is fairly 
easy to learn. 

B. Videodisc 

The optical videodisc required by InfoWindow contains up to 54,000 frames, 
each with its own address. Displaying 30 frames/sec, the disc contains up to 30 
min of playing time. On our equipment (using a Pioneer LD-V6000 videodisc 
player) the access time from one frame to any other is approximately 1 .6 sec 
maximum. Our videodisc was pressed from a 3/4" videotape, precisely the tape 
used in the videotape instructions group. Its length was 27 min. 

ER?C 8 
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C. Stimulus Materials 

Both the passive videotape instructions and the interactive ones were 
intended to teach the viewer to build the lift. The videotape instructions were 
designed as follows. (See Baggett, 1983, 1985; Baggett & Ehrenfeucht, in press, 
for details.) A group of subjects built the lift a total of 47 times, copying from 
a physical model. The order in which each subject selected the pieces for 
assembly was recorded. Using a computer package for cluster analysis by Perry 
(1983), the "most typical" conceptualization (breakdown into parts, subparts, 
etc.) among the 47 was selected. This hierarchical structure, given by a single 
subject, is shown in Figure 2 as an ordered tree. (Seventy percent of subjects' 
conceptualizations were minor variants of the "typical" one.) A videotape 
showing assembly of the lift was made, based on the typical hierarchical 
structure and on the particular subject's order of selecting pieces. The 
videotape was shot top down, depth first, and the steps were shown in an 
executable order. Two cameras were used, one showing the current goal (or 
subassembly) and the other showing hands working toward the goal. In addition, 
each subassembly (for example, the string guide ) contained a two-part sequence: 
(1) get the pieces needed; and (2) assemble the part. The instructions were thus 
modularized into "natural" conceptual units.'' 



Insert Fig. 2 about here. 



The videotape instructions were also "optimized"" according to two other 
parameters analyzed in previous studies: 

(1 ) The verbal descriptions (names) used in the narration have been shown to be 
better than some others. They were selected from collections of names 
generated by subjects. See Baggett, Ehrenfeucht, and Perry (1986) for the 
procedure. 

(2) The temporal overlap of visual material and spoken narratioin leads to good 

o 
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associations between names for objects or actions and their pictorial referents 
(Baggett, 1984): tiie visual material precedes, or is in synchrony with, the spoken 
material; spoken does not precede visual. 

The optimization of conceptualization and sequencing, naming, and temporal 
overlap occurred in the video presentations of all six groups tested. Thus it 
would have been easy to make presentations such that subjects' performance was 
worse. However, the task of building the lift is so difficult that the average 
observed performance is only 50 to 60% of the possible performance. No subject, 
working for 90 or 100 minutes, could learn to do the task perfectly. So we 
manipulated factors to try to get some significant difference in performance. 
But large differences cou/d not be expected. 
D. Design of the Interactive Presentation 

The interactive instructions were designed using the same images and 
narration found on the videotape. (The videotape was pressed into a videodisc.) 
The design was as follows: 

• Just as with the videotape, the assembly was modularized into 

conceptual units, showing a step-by-step executable procedure, with 
"get pieces" and "assemble part" subunits. 

• The only input permitted from the subject was touches to the touch 
screen (no keyboard input). From touches, a viewer could manipulate 
what was seen. 

• The viewer was forced to see all information once. (No information 
could be skipped.) The subject could not skip around in the 
presentation but had to see the assembly in a predetermined 
executable order. 

• Figure 3 shows the structure of the interactive presentation. Options 
available to the viewer are shown as arrows. The presentation 
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stopped at the end of each conceptual subunit (e.g., "get" or 
"assemble"), giving a still frame. The subject could continue or replay 
with a touch. (Words used as instructions for touches appeared at the 
bottom of the screen as text over moving or still video.) For example, 
after watching the assembly of the crank handle, the viewer could 
replay "assemble crank handle,' or both "get pieces and assemble 
crank handle." Or the viewer could go ahead to "get pieces for string 
guide." 



Insert Fig. 3 about here. 



• During replay of a unit, the viewer could touch the word "next" and skip 
the rest of the replay, moving to the next unit. 

• Subjects could stop the video at any time, by touching "pause." This 
gave a still frame. They could resume the video by touching "resume." 

• The presentation was not menu-based. That is, while choices were 
available, they were not hierarchically arranged, and they were 
typically of the form "next" or "replay." (See Fig. 3.) 

III. Theoretical Predictions 

The main purpose of the experimental work was to determine the role 
of practice in interactive multimedia instruction.^ Namely, for good 
performance and retention of a procedure, should a person practice while 
being (interactively) taught the procedure? 

In our theoretical framework, concepts are the basic units used for 
thinki.ng. Our formulation (Baggett & Ehrenfeucht. 1982, 1985; see also 
Baggett, Ehrenfeucht, & Hanna, 1987) assumes that concepts are 
multimedia. This means that information that is put together into a 
concept comes from different sources, for example, visual and auditory. 
It also means that a concept contains other information, such as motoric 
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information. An example of motoric information is information about 
flow to move one's iiand, or how to put two blocks together. 
Processor^are assumed to create and use concepts. We hypothesize 
that there is one processor, which we call central, whose main task is to 
form and modify concepts. Other processors provide values for concepts. 
For example, the visual processor might provide an encoding of size or 
shape or color.. The linguistic processor might provide a verbal label. And 
the motoric processor might provide instructions which direct 
movement, such as sneeze, move the left hand, or chew. 

In an assembly task, the central processor ideally forms the concept 
of how to build the object. The visual processor provides information 
about how pieces of the object look, and their configuration. The 
linguistic processor provides names of pieces and subassemblies. And 
the motoric processor provides information about how to put pieces 
together. Thus in our tasks, information comes from four processors: 
central (the concept of building the object), motoric, from practice (one 
does it), visual, from practice and from vi^-eo (one sees it), and verbal, 
from narration (one hears it). We have interpreted our experimental 
work using this framework. For example, in Baggett, 1987, we 
hypothesized that integrated concepts (concepts with motoric, visual, 
and linguistic elements linked together as subconcepts of the abstract 
concept of how to build the object) are best for procedures, leading to 
good performance from memory and good retention over a delay. In the 
stuc^* '~ome subjects watched a videotape showing assembly of a toy 
h( wUlle others first built the helicopter using a model as a 

g chen watched the videotape (or first watched the videotape 

and then built from a model). All three groups were then tested from 
memory. The groups who built during training performed more than 
twice as well (on both structural and functional measures) as the group 
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who did not build during training. Furtlier, during instruction, tlie groups 
who built were given all three conceptual elements, motoric, visual, and 
verbal (motoric and visual from building; visual and verbal from the 
videotape). The group who did not build was given only visual and verbal 
elements (from the videotape). Using our multimedia framework, we 
interpreted this to mean that when instruction provides motoric, visual, 
and verbal elements, they are automatically integrated into one concept, 
i.e., motoric, visual, and verbal elements become subconcepts of the 
abstract concept of building the object and such a concept gives good 
performance from memory. 

We schematically represent the situation as shown in Fig. 4, part I. 
The abstract concept A of building the object has three subconcepts, B, ' 
C, and D, as indicated on the left by solid arrows. B is visual 
information, C is linguistic, and D is motoric. When all three are present 
in instruction, we hypothesized, they become subconcepts of the same 
concept, namely, they are chunked as a whole. (On the right of Fig. 4 is 
the same situation, but drawn as a Venn diagram.) 



Insert Fig. 4 about here. 



Other situations are possible in our framework, however. Besides the 
subconcept relation between concepts, there is also a pointer or 
association relation, which we indicate by a dotted arrow. The pointer is 
interpreted as causality, expectation, or temporal contiguity. Suppose, 
for example, that motoric information (gained from practice) is not 
integrated into the concept as a subconcept, but only associated to it. 
This situation is schematically represented in Fig.4, II. What 
performance would we expect if such a conceptual structure is built? 
First, a concept and its subconcepts, as in Fig. 4, 1, forms a chunk and 
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can be processed as a unit. But information that is only associated, as in 
Fig. 4, II, is not chunked with the information that points to it. It is 
likely that one can activate the concept A but not be able to follow the 
pointer to D. Intuitively, the subconcept relation brings with it 
annexation, while the association relation provides only a path. 
IV. The Experiments 

In our first experiment in this study, we compare performance of 
people who build the lift from memory after watching the videotape to 
performance of those who build from memory after interactive 
instructions during which they actually build the lift. If the above 
theoretical hypothesis is correct, that motoric, visual, and verbal 
elements automatically become subconcepts of the same concept when 
they are present in interactive instruction with simultaneous practice, 
then the prediction is straightfonward: Interactive instruction with 
building on-line should be better than passive video. The first allows 
integrated concepts; the second provides no motoric component. 



Method 

Subjects. Sixty-four students, 32 of each gender, participated as part 
of the University of Colorado Psychology 100 course requirement. Half 
(16 of each gender) were assigned to the videotape group and half to the 
interactive build-on-line group. 

Design. The design was a 2 (instructions) x 2 (gender) between subjects 
ANOVA. 

Stimulus materials. The videotape and interactive videodisc-based 
presentations described above were used. 

Procedure. Subjects were run individually. First they filled out a short 



questionnaire asking their major, native language, and an estimate of 
their experience with assembly kits. They were told, "Today you're going 
to be given instructions on how to build a fairly complicated object from 
a kit of pieces. After the instructions, you'll be asked to build the object 
from memory, so try to learn as much as you can from the instructions. 
You will be instructed via videotape (one group) or by using an 
interactive videodisc-based computer system (the other group)." 

Subjects then performed a matching task, with the following 
instructions: 

"Rrst, to get you accustomed to the pieces in the assembly kit and 
their names, please do this matching task. You have before you a 
collection of one of each of the 48 different pieces in the kit and some 
sheets of paper containing the 48 names. Spread the sheets out in front 
of you and put each piece by its correct name. This is not a test. If you 
have trouble or want assistance, just ask the experimenter, and she (or 
he) will help. When you're done, the experimenter will check your 
matches and correct any you missed." 

The matching task took about 5 min. 

Subjects in the videotape group were positioned before a 15 in. color 
tv monitor and reminded to learn as much as possible for the memory 
trial. They placed their chair across from the monitor at any distance 
they wanted, and the 27 min tape was started. 

After the tape was completed, they moved to another table to begin 
the memory trial. They were reminded to build a lift as much like the 
one they had seen built as possible, and they were told there was no time 
limit. The experimenter had a kit of pieces, and the subject was required 
to ask for them one-by-one, either by name or by pointiiig. The subject 
had a folder with color photos of the pieces and their names, to aid in 
getting the pieces. The subject was to!d that pieces did not have to be 
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used once they were requested; they could be left aside and never used if 
the subject desired. The experimenter recorded the time the subject 
requested the first piece and the time the subject quit, and the order in 
which the subject requested the pieces. 

(A subjective observation of experimenters testing subjects from all 
groups was that they very much enjoyed doing the task. They typically 
started off quite accurately, and when their memories began to fail, they 
would create, making comments such as, "I can make a lift better than 
the one in the instructions." Experimenters were encouraging and 
positive to all subjects, and subjects did not seem to lack motivation, 
although this was not objectively measured.) 

Subjects in the interactive videodisc-based group, after doing the 
matching task, sat before the InfoWindow monitor. The experimenter 
explained that the presentation showed a step-by-step procedure for 
assembling the object, and that the subject could have some control over 
what was shown by touching labels on the touch screen. The labels were: 
next, pause, short replay, long replay, very long replay, and replay whole 
presentation. The meaning of each was explained. The subject was told 
to build the object once while watching and to prepare for a memory 
trial im.mediately afterwards. He or she was told that there was no time 
limit; the instructions could be reviewed as long as the subject wanted, 
and played again from the beginning if the subject desired. The subject 
could build at any time: while the video was moving, or while there was a 
still frame. During the presentation, subjects were required to ask for 
pieces one-by-one. (They had folders with photos and names of pieces as 
above.) The experimenter provided the pieces and typed in their 
abbreviated names as they were requested. These names and all touches 
made by subjects were automatically recorded (with time stamps) on a 
log file provided by IBM's Composer/Conductor software. 

EMC jg 
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The procedure for these subjects' memory trial was identical to that 
described above for videotape subjects. 

Results 

Scoring.. An abstract graph of each subject's lift built from memory 
was drawn. Pieces were drawn as nodes, and physical connections were 
drawn as links. A graph of the correctly assembled lift is shown in 
Figure 5. (The solid and dotted lines in Fig. 5 correspond to the sub- and 
subsubassemblies shown in Fig. 2.) Nodes are numbered to correspond to 
particular pieces in the lift, as shown in Figure 1. To score a subject's 
lift, the number of correct connections was counted. (There are 1 04 
connections in the correctly built lift.) For example, is node 1 connected 
to node 4? 2 to 4? 2 to 80? This measure teils how similar in 
structure the lift built from memory Is to the correctly assembled lift. 
In addition, a functional score, either yes or no, was taken?»-Does the lift 
work? Namely, can something be turned resulting in a carrier going up 
and down a tower? (Lifts were scored by two people, and when there 
was disagreement, a third person broke the tie.) We first note that not 
one subject in any group built a perfect lift from memory. 



Insert Fig. 5 and Table 1 about here. 



Comparison of Interactive vs. Passive Instruction. Rows 1 and 2 
of Table 1 give memory performance for both structure and function in 
the two groups, broken down by gender. (Measures of time and 
efficiency, and number of pieces used in the memory trial, discussed 
below, are also given.) First, there are no gender differences. But more 
surprisingly, there are no significant differences in structure or function 
between the group viewing the videotape and the group receiving 
interactive instructions and building on-line! (F values < 1 in all cases: 
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statistics are reported below.) 

The question is, why is interactive instruction with building on-line 
not better than passive video? Why can't people listen and watch and do 
simultaneously? Baggett & Ehrenfeucht, 1983, showed that people can 
listen and watch very well simultaneously, e.g., a movie. But here we 
find that when practice is added, there is no additional benefit. The 
theoretical hypothesis was that when instructions provide motoric, 
visual, and verbal components, they automatically sit together in the 
same concept (become subconcepts of the same concept), as in Fig. 4, 1. 
Hence performance and retention of a procedure are best. What 
modification of the hypothesis is needed? We note that the interactive 
instructions with building on-line are quite a different situation from 
the helicopter experiment reported above, in Baggett (1987). For 
example, here practice is mixed with video; above they were entirely 
separate. 

Interactive Instr? - jtlons with No Building On-line 



We first hypothesized that the interactive-build group above had a 
dual motoric task (build the lift and operate the touch screen). 
Perhaps, we thought, operating the screen interferes with learning to 
build. Therefore we hypothesized that subjects who used the interactive 
instructions as above (i.e., operated the screen) but who were not 
allowed to build on-line would perform even worse from memory than 
either the group with video alone or the group building on-line: the 
motoric processor would have a task (operate the screen), but the 
movements required are not related to learning to build the lift. 
Thirty-two subjects were run as before. They were given interactive 
instructions but were not allowed to build on-line. They built only once 
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from memory, after their interactive trial. 

The results from this group, termed interactive no-build, are given in 
Row 3 of Table 2. They perform best of all, although not significantly so. 
We define a new measure, efficiency, as the number of correct 
connections per minute of total time to work. (Total work time = study 
time + memory trial time.) Using the efficiency measure, the interactive 
no-build group (efficiency = .645) is significantly better than the 
interactive build group, efficiency = .51 (t(90 df) = 2.96, p < .01 ; Winer 
(1971), p. 385), and significantly better than the passive video (.565) and 
interactive build groups together (multiple comparisons, F(1 ,90) = 5.86, 
p < .05), but not significantly better than the passive video group alone 
(t(90 df) = 1 .93, n.s.). 

The fact that the no-bui!d group performs as well as the build group on 
structure and function, and better on efficiency, is both theoretically and 
practically interesting. For this task, motoric elements during 
interactive instruction do not help later performance, and, when the 
measure is efficiency, actually hinder performance! Wo note that the 
motoric elements required in the task of building the lift are actually 
known to all subjects: everybody can join together two blocks. What 
subjects must do is associate known motoric actions with the task of 
building the lift. This situation is different from, for example, learning 
to play tennis, where people have to learn basic motoric skills. In 
tennis, the most important element is practice. 



Interactive Instructions with Building while Screen is Black 

Thus far, it appears that motoric elements given during training are 
not integrated into the task of building the lift. We next hypothesized 
that the problem for the interactive-build group was one of divided 

ER?C .19 



page 1 4 

attention for the visual system: One must watch the screen, and watch 
one's hands while building. Perhaps this dual visual task was causing the 
decrement in performance. So we ran a fourth group of subjects as 
follows. Subjects were given interactive instructions and required to 
build on-line, but they could build only when the screen was black . A 
small modification was made to the interactive instructions used 
previously. A new touch area, labeled "build," was added. Whenever the 
subject desired to build on-line, he or she was required to touch "build," 
and the screen went black, with the word "return" in the corner. When 
the subject finished building, he or she touched "return" and the screen 
returned to the exact frame at which it had gone black. The subject 
could then check what had been built, and if changes were needed, "build" 
had to be touched again. 

In the black screen presentation, at a given moment, a viewer's 
attention is not divided. One watches the screen or watches one's hands, 
but not both. The subject actually performs a memory trial during the 
interactive trial. Namely, the subject builds, working only from what he 
or she has in his or her head, without a picture. The subject actually 
gets less instruction , in the sense that instruction is taken away! Also 
the subject engages in problem solving during the interactive trial. That 
is, the subject must figure out what to do; in the other interactive build 
group, the subject could merely copy from the screen. 

Forty-four subjects (22 of each gender) were run in the black screen 
group. The procedure was identical to the other interactive groups, 
except subjects were told that they could build on-line only when the 
screen was black, and about how to make the screen black and to bring it 
back to normal using the touch screen. They were told to prepare for a 
memory trial after their interactive trial, and immediately after the 
interactive trial, they built the lift from memory. 
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Table 2, Row 4 gives the results on the memory trial for the black 
screen group. Building only when the screen is black helps males -- they 
score 64% on structure from memory. This is highest of any group, but 
not significantly higher than the interactive-no build males, who score 
61 .2%. But the black screen instruction hurts females - they score only 
49% on structure, lowest of any group, but not significantly lower than 
the interactive-build females (51%) or the passive video females (51%). 
Overall, the black screen presentation leads to slightly but not 
significantly worse performance than the interactive-no build 
presentation, and for the first time there is a significant gender 
difference, with males performing better than females, t(42 df) = 2.39, 
p< .03. 

Thus far, then, we have not found a presentation condition in which 
practice is included in instruction and performance on a later memory 
trial yields significantly better structural or efficiency scores than 
when practice is not included! 

Interactive Instructions with 7-day Delay Between Training 

and Test 

In the toy helicopter experiment of Baggett (1987), people who 
practiced during training far outperformed those who received only 
audiovisual instruction, when the test was a later memory trial. (Their 
practice, as discussed above, was copying from a model; it was not 
practice mixed with interactive instructions.) Further, when a 7-day 
delay was put between training and test, the difference in favor of the 
group who practiced was even greater. Theoretically, the result showed 
that retention of a procedure for assembly is especially poor when no 
motoric component is provided in the original instruction. 
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In one last attempt in (his study to find an experimental condition in 
which practice during interactive instruction gives an advantage over no 
practice, we ran two more groups. They were identical to groups 2 and 3 
in Table 1 (interactive-build and interactive-no build), except a 
one-week delay was put between their training and test sessions. 

Thirty-two subjects (16 of each gender) were run in each of the 7-day 
delay groups. The procedure was the same as before; one group built 
on-line during instruction, and the other did not. Both were told to 
prepare for a memory trial session 7 days later. Rows 5 and 6 of Table 
1 give the results. 

Two ANOVAs were performed, on both structural and efficiency 
measures. Each was 2 (gender) x 2 (instruction: build vs. no build) x 2 
(delay: 0 vs. 7-day). For the ANOVA analyzing structure, delay was the 
only significant effect, F(1 ,120) = 21 .8, p < .001 . The ANOVA on 
efficiency yielded two significant effects. First was instruction, 
F(1 ,1 20) = 9.1 , p < .01 ; delay was also highly significant, F(1 ,1 20) = 61 .9, 
p < .001. Thus, the no build groups perform better than the build groups 
on efficiency, and those tested at zero delay perform better than those 
tested after a week. 

The results are fairly straightforward to interpret. Once again, there 
is no advantage to practicing during interactive instruction: overall, the 
no build groups perform as well as or better than the build groups. On 
structural scores, females tested after a week average 43.4 (build) and 
33.6 (no build). A two-sample t-test determined that the difference was 
not significant, t(30 df) = 1 .52. 

The last column in Table 1 shows the m.ean number of pieces requested 
during the memory trial. It varies from 63 to 74, with the 0-delay build 
and no build groups averaging 68 and 74 respectively, and the 7-day delay 
build and no build groups averaging 64 and 67 repectively. These 



differences show that the build groups have good visual recognition of 
the pieces needed in the lift. While they do not score higher on structure 
or function than the no build groups, they are somewhat better at picking 
out the correct pieces. Thus practice during interactive instruction 
gives a small (but not significant) advantage: piece selection is more 
accurate. 



Insert Figs. 6 and 7 about here. 



Figure 6 shows percentage correct connections for the 0-delay build 
and no build groups as a function of section of the lift worked on. The 
X-axis lists the subassemblies of the lift in the order in which they were 
presented in the instructions. For example, subassembly 1 is 
T-platform, subassembly 2 is stopper, etc. (See Figure 3 for the order.) 
Figure 7 shows the 7-day delay data, divided by gender. These graphs 
show that performance within a subassembly is quite stable over groups, 
that subassemblies vary considerably in their difficulty, and that 
performance gets steadily worse the later in instructions it is taught. 
One reason for the degradation, of course, is that later connections in the 
lift are sometimes dependent on earlier ones. For example, one cannot 
correctly connect the column to the lift base unless certain parts of the 
lift base are built correctly. 

Discussion 

When this experimental work was begun, the expectation was that 
practice during interactive procedural instructions ought to aid in 
learning and retaining the procedure. Yet we found, in a series of six 
different experimental conditions, that groups who practiced never 
significantly outperformed those who did not practice, and sometimes 



the practice groups performed significantly worse.^ 

The procedure to be learned, assembling an 80-piece object from a kit 
of pieces, 's quite lengthy but does not take much skill. For example, 
people building the object from a model can copy it perfectly without 
much difficulty. Thus the problem is not in acquiring the skill to 
perform the actual movements. Rather, our view is that one must link 
together as a subconceptthe movements required and the concept of 
building the object. It appears that this subconcept linkage is not 
formed when practice is mixed with interactive instruction. Rather, we 
hypothesize that the situation is like that shown in Fig . 4, II: the 
motoric components are merely pointed to. However, it does happen 
when practice occurs mixed sequentially with video instruction 
(Baggett, 1987). Namely, the situation is like that shown in Fig. 4, 1: the 
motoric elements are a subconcept. In order to change the linkage of the 
motoric components from pointer to subconcept, as shown in Fig. 4, III, 
we hypothesize that one of two things must happen: (1) practice must be 
^oquentially and not simultaneously arranged with audiovisual 
instruction; or (2) there must be more extensive practice, as we discuss 
below. 

Related Explanations 

We offer here three related interpretations of the findings of this 
study. In our framework of multimedia concept formation, different 
processors can work simultaneously. The interaction between 
processors can be competitive or cooperative. If it is competitive, two 
processors share some resources (Norman and Bobrow, 1975) and 
compete for who is going to control the resource. If it is cooperative, 
one processor has information that is needed for another, and simply 



provides it. It as if tiie processor says, "I already prepared this--you can 
use it." 

1 and 2. Overloading a Processor or Sharing a Resource 

In the task given to our subjects, we have .hypothesized that several 
processors are involved, and we have also hypothesized about what each 
is doing. When performance is better than one would expect in the 
situation, it is an example of cooperation. When it is worse, we look at 
two possible interpretations: 

1 . Overloading of one of the processors: One processor is doing more 
than expected, and therefore there is a bottleneck. For example, the 
results of the interactive build group (2 in Table 1) made us suspect that 
the motoric processor was overworked. It is involved in hand 
movements (operating the screen and building the lift) and in eye 
movements (directing the eyes). The "extra" task of building on-line 
could have created an overload. In our framework, if a processor is 
overloaded, its needs exceed the resources available. For example, the 
processor cannot do the task in the time allotted, or it cannot do it with 
the memory allotted. So something must go. 

In forming a multimedia concept, the central processor creates a 
motoric node. The motoric processor provides a value for the node. That 
is, during encoding, the motoric system prepares a value that is stored. 
It might be that when one practices on-line, the motoric system does not 
prepare a value that is stored. Instead, it keeps track of what the person 
is doing, and this prevents it from providing a value that goes into the 
node. 

To put it another way, movements seem to distract one from paying 
attention. A possible explanation is that any kind of coordinated 
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physical movement requires a large amount of information to pass 
between the brain and peripheral nervous system. This means, 
informationally, that the system is working at full capacity. Therefore 
there is no room for passing other information; there is too much traffic. 

A hypothesis, then, is that there is a three-part scheme: plan, execute, 
reflect. Perhaps learning and planning take part before or after the 
action, but not during. 

Or it could be that the visual system is overloaded, here we 
hypothesize the same mechanism as above, but now the motoric system 
is putting an extra requirement on the visual system, to look and keep 
track. 

Theoretically, these two hypotheses could be distinguished. The first 
(the motoric system is overloaded and does not provide a motoric value) 
implies that there would be poorer performance in speed or accuracy 
when the person was presented with a visual or verbal stimulus and told 
to do the task. The second (visua'i system overloaded) implies that a task 
involving just visual recognition would be impaired. As we noted above, 
visual recognition (ability to choose the right pieces) is slightly better 
in the build groups, indicating that if there is an overload, it probably 
lies with the motoric processor. 

2. If two processors are in competition, one could be impaired because 
of having to share a resource. This hypothesis is difficult to distinguish 
from the previous one, because it is not easy to specify what a shared 
resource might be. 

3. Timing and Synchronization 

Another explanation involves timing. Timing could be a problem if, 
from the point of view of each processor, there is no difficulty, but 
passing information between processors, i.e., communication, is out of 
synchrony. Within the model, no processor is overloaded. Each element 
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(node and value) in the multimedia concept formation is created correctly, 
but they're not assembled correctly. The elements are satisfactory, but 
the graph formed is not. 

This hypothesis we think is worth pursuing, because, if it is true, 
following further the black screen idea, to force a person to perform 
specific tasks with specific timing, could lead to very strong differences 
in performance. Another reason it is worth pursuing is that it is 
optimistic. It means people are good information processors if 
information is organized correctly. It leaves opsn the possibility that one 
can learn by doing. 

Subjects in the no build groups above were not given practice during 
training. But it is possible that .ney constructed motoric values for their 
concepts based on what they were shown, i.e., they imagined the (correct) 
movement. Subjects in the build groups were given actual practice, which 
they might have stored as motoric values. But they did not perform better 
that the no build groups. People building on-line did nci always build lifts 
perfect in every detail, and there were certainly moves made in on-line 
assembly that had to be corrected. When one is not satisfied with 
practice, one may have a conflict between how it should be done and how 
one did it. What would one remember from training? 

If hypothesis three is correct, then there is a definite prediction. If the 
subject is allowed to practice until he or she is proficient, then practice 
should not interfere and should help. If the subject is forced to quit 
before he or she is proficient, then practice is harmful. Partial 
knowledge may be worse than none. 

We certainly do not consider that this study settles the issue of the role 
of practice in interactive procedural instructions. But it does dispel one 
popularly held intuition: that on-line practice always gives better 
performance. 

ER?C ^'7 
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After running the passive video group, we learned that the particular 
videotape used resulted in significantly better performance than did two 
other videotapes showing the lift's construction that we have used in 
previous work (Baggett & Ehrenfeucht, in press). 

'^Since our interest was the role of practice in interactive instruction, we 
did not compare performance from many other possible groups, e.g., a group 
whose training consists of simply copying from a physical model (as in 
Baggett, 1987). 

"^In our framework (Baggett & Ehrenfeucht, 1982), a processor is a partial 
function from a state of memory and some input into a state of memory 
and possibly some output. It changes the state of memory and provides 
interaction with the environment. 

^It is possible that some other experimental arrangement, such as having 
only the correct pieces available during training and/or test, would lead to 
better perforinance by groups that practice. Our framework does not make 
a specific prediction about this; the question could be answered 
empirically. 
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Figure Captions 



1 . An 80-piece object, called a lift , made from the Fischer-Technik 
assembly kit. 

2. "Typical" breakdown of the lift into sub- and subsubassemblies. This 
hierarchical tree was derived from data from humans, as explained in 
the text. 

3. Design of the interactive videodisc-based instructions for asL'smbly of 
the lift. Arrows indicate options available to subjects via touches to 
labels on the touch screen. Touching "next" took a subject to the next 
unit. "Short replay" replayed the unit just viewed. "Long replay" 
replayed the previous two units. "Extra-long replay" replayed an entire 
subassembly. And "replay whole presentation" replayed from the 
beginning. 

4. Three possible relationships among concepts and subconcepts, as viewed 
in our multimedia framework. I shows a concept A with 3 
subconcepts, B (visual), C (linguistic) and D (motoric). II shows A with 
2 subconcepts, B and C; A is associated to (pointing to) D, but D is not a 
subconcept of A. Ill shows the incorporation or annexation of D as a 
subcohcept of A. 

Abstract graph of the lift shown in Fig. 1 . Nodes indicate pieces in the 
lift, and links indicate physical connections. The numbers in Fig. 1 
correspond to the identically numbered nodes in Fig. 5. The graph is 
divided into the same subassemblies (solid lines) and subsubassemblies 
(dotted lines) given in Fig. 2. 

Mean percentage correct on each subsubassembly when building from 
memory for the four groups tested at zero delay, as a function of when 
the instruction was presented. A linear regression analysis yields y = 
85.6 - 4.1 4x, R = 0.72. 

Mean percentage correct on each subsubassembly when building from 
memory for the groups tested after a 7-day delay, as a function of 
gender and when the instruction was presented. A linear regression 
analysis yields y = 65.8 - 3.4x, R = 0.62. 

SI 
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Table 1. 



Data from six groups of subjects who built the lift (shown in Figure 1 ) from memory after different types of 
instajction and different delays. 



meontime 
taken for 





mean% 
correct 
connections 
inSftbuat 
from memory 
(lift has 104 
c o nr^e c tio n s) 


number of 

tiinctional 

tftsbuilt 

from 

memory 


training trial 
(passive video 
group: tape 
lengtti; ottier 
groups: 

trial lengtfi 
(minutes) 


meantime 
taken for 
performance 
from memory 
^rnernory 
trial) 

(minutes) 


mean total 
time for 
instruction 
(training + 
memory) 
(mhutes) 


mean 
overaB 
efficiency 
(% correct 
connectiorc 
+ total tme) 


number of 
pieces 
requested 
in memory 


group: 
















video 

(32 subjects) 




I / OT 0^ 


2.1 


64.95 


91.95 


.565 


73.45 


males (16) 


49.4 


9 of 16 


27 


70.8 


97.8 


.53 


76.8 


femoles(16) 


51.4 


8 of 16 


27 


59.1 


86.1 


.60 


70.1 


2. interactive- 
build 

(32 subjects) 


52.2 


16 of 32 


7 




lOo.l 


.51 


66.5 


males(16) 


53.1 


9of 16 


60.1 


>19 O 

4^.7 




.51 


69 


females (16) 


51.3 


7 of 16 


59.3 


44.0 


103.3 


.51 


64 


3. interactive- 
no-bu3d 
(32 subjects) 


59.5 


19 of 32 


44.25 


49.6 


93.85 


.645 


74.7 


males (16) 


61.2 


12 of 16 


43.6 


49.3 


92.9 


.0/ 


7ft 


females (16) 


57.8 


7 of 16 


44.9 


49.9 


94.8 


.62 


71.4 


4. interactive- 
buildwhen 
screen diuck* 
0 deloy 
(44 subjects) 


56.9 


19 of 44 


72.3 


45.25 


117.55 


.50 


69.45 


males (22) 


64.2 


17 of 22 


70.2 


43.4 


113.6 


.59 


74.7 


femoles (22) 


49.4 


7 of 22 


74.4 


47.1 


121.5 


.41 


64.2 


5. hteractive- 
build-7day 
(32 subjects) 


42.85 


19 of 32 


62.7 


83.9 


146.6 


.295 


63.75 


males(16) 


42.3 


11 o. 16 


56.7 


82.1 


138.8 


.30 


66.2 


femoles (16) 


43.4 


8 of 16 


68,7 


85.7 


154.4 


.29 


61.3 


6. interactive- 
no bu3d- 
7 day 

(32 subjects) 


40.6 


19 of 32 


43.0 


82.0 


125.0 


.355 


66.75 


males (16) 


47.6 


12 of 16 


40.6 


82.0 


122.6 


.43 


72.6 


females (1 6) 


33.6 


9 of 16 


45.4 


82.0 


127.4 


.28 


60.9 




Figure 2 



percentage correct as function of instructions and order 




1 2 3 4 5 6 7 8 9101112131415 
subassembly number 



o 40 
ERIC 



Figure 6 



41 



percentage correct as function of instructions, gender, and order 
100 T ■ , 




• • ■ • ■ I r I I I I I I 1 

123456789 101112131415 



subassembly number 




Figure 7 



a 



Dr. Beth Adelson 
Department of Computer 
Science 
Tofts University 
Medford, MA 02155 

Personnel Analysis Division, 

AF/MPXA 
5C360, The Pentagon 
Washington, DC 20330 

Air Force Human 
Resources Lab 
AFHRL/MPD 
Brooks, AFB, TX 78235 

AFOSR, 

Life Sciences Directorate 
Boiling Air Force Base 
Washington, DC 20332 

Technical Director, 

Army Human Engineering 
Lab 

ATTN: SLCHE-D 
Aberdeen Proving Ground 
MD 21005-5001 

Dr. Robert Ahlers 
Code N71 1 

Hiiman Factors Laboratory 
Naval Training Systems 
Center 

Orlando, FL 32813 

Dr. John R. Anderson 
Department of Psychology 
Carnegie-Mellon University 
Schenley Park 
Pittsburgh, PA 15213 

Technical Director, ARI 
5001 Eisenhower Avenue 
Alexandria, VA 22333 

Dr. Heinrich Bauersfeld 
IDM der Universitaet 
Postfach 8640 
Bielefeld 

WEST GERMANY D-4800 

Leo Beltracchi 
United States Nuclear 



Regulatory Commission 
Washington DC 20555 

Dr. Jeff Bonar 
Learning R&D Center 
University of Pittsburgh 
Pittsburgh, PA 15260 

Dr. Robert Breaux 
Code 7B 

Naval Training Systems 
Center 

Orlando, FL 32813-7100 

Dr. John T. Bruer 
James S. McDonnell 
Foundation 
Suite 1610 

1034 South Brentwood Blvd. 
SL Louis, MO 63117 

Dr. Joanne Capper, Director 
Center for Research into 
Practice 

1718 Connecticut Ave., 
N.W. 

Washington, DC 20009 

Dr. John M. Carroll 
IBM Watson Research Center 
User Interface Institute 
P.O. Box 704 
Yorktown Heights, NY 
10598 

CDR Robert Carter 
Office of the Chief 

of Naval Operations 
OP-933D4 

Washington, DC 20350- 
2000 

Dr. Fred Chang 

Pacific Bell 

2600 Camino Ramon 

Room 3S-450 

San Ramon, CA 94583 

Dr. Raymond E. Chrif J 

UES LAMP Science Advisor 

AFHRL/MOEL 

Brooks AFB, TX 78235 

Dr. William Clancey 



Institute for Research 
on Learning 
3333 Coyote Hill Road 
Palo Alto, CA 94304 

Assistant Chief of Staff 

for Research, 
Development, 

Test, and Evaluation 
Naval Education and 

Training Command (N-5) 
NA? Pensacola, FL 32508 

Brian Dallman 

Training Technology Branch 
3400 TCHTW/TTGXC 
Lowry AFB, CO 80230- 
5000 

Dr. Denise Dellarosa 
Psychology Department 
Box HA, Yale Station 
Yale University 
Mew Haven, CT 065?0- 
7447 

Defense Technical 

Information Center 
Cameron Station, BIdg 5 
Alexandria, VA 22314 
Attn: TC 
(12 Copies) 

Dr. Jean-Pierre Dupuy 
Ecole Polytechnique 
Crea 1 Rue Descartes 
Paris, FRANCE 75005 

ERIC Facility-Acquisitions 
4350 East-West Hwy., Suite 
1100 

Bethesda, MD 20814-4475 

Dr. Marshall J. Farr, 
Consultant 

Cognitive & Instructional 
Sciences 

2520 North Vernon Street 
Arlington, VA 22207 

Dr. Paul Feltovich 
Southern Illinois University 
School of Medicine 



44 



Medical Education 
Department 
P.O. Box 3926 
Springfield, IL 62708 

Dr. J. D. Fletcher 
Institute for Defense 
Analyses 

1801 N. Beauregard St. 
Alexandria, VA 22311 

Dr. Barbara A. Fox 
University of Colorado 
Department of Linguistics 
Boulder, CO 80309 

Dr. Michael Friendly 
Psychology Department 
York University 
Toronto ONT 
CANADA M3J1P3 

Julie A. Gadsden 
Information Technology 
Applications Division 
Admiralty Research 
Establishment 

Portsdown, Portsmouth P06 
4AA 

UNITED KINGDOM 

Dr. C. Lee Giles 
AFOSR/NE, BIdg. 410 
Boiling AFB 

Washington, DC 20332 

Dr. Sarr, Glucksberg 
Department of Psychology 
Princeton University 
Princeton, NJ 08540 

Dr. Sherrie Gott 
AFHRL^MOMJ 
Brooks AFB, TX 78235- 
5601 

Dr. T. Govindaraj 
Georgia Institute of 

Technology 
School of Industrial 

and Systems Engineering 
Atlanta, GA 30332-0205 

Dr. Gerhard Grossing 



Atominstitut 
Schuttelstrasse 115 
Vienna 

AUSTRIA A-1020 



Dr. Ronald K. Hambleton 
University of Massachusetts 
Laboratory of Psychometric 
and Evaluative Research 
Hills South, Room 152 
Amherst, MA 01003 



Dr. Wayne Harvey 
Center for Learning 
Technology 

Education Development 
Center 

55 Chapel Street 
Newton, MA 02160 



Dr. James Hollan 
NPRDC, UCSD 
Code 501 

San Diego, CA 92152 



Dr. Janet Jackson 
Rijksuniversiteit Groningen 
Biologisch Centrum, VIeugel 
D 

Kerklaan 30, 9751 NN Haren 
The NETHERLANDS 



Dr. Robin Jeffries 
Hewlett-Packard 
Laboratories, 3L 
P.O. Box 10490 
Palo Alto, CA 94303-0971 



Dr. Douglas H. Jones 
Thatcher Jones Associates 
P.O. Box 6640 
10 Trafalgar Court 
Lawrenceville, NJ 08648 



CAPT Tom Jonos 
ONRCode12lD 
800 N. Quincy Street 



Arlington, VA 22217- 
5000 

D\. Sylvan Kornblum 
University of Michigan 
Mental Health Research 
Institute 

205 Washtenaw -Place 
Ann Arbor, Ml 48109 

Dr. Benjamin Kuipers 
University of Texas at 
Austin 

Department of Computer 
Sciences 

Taylor Hall 2.124 
Austin, Texas 78712 



Dr. Robert W. Lav/ler 
Matthews 118 
Purdue University 
West Lafayette, IN 47907 



Dr. Alan M. Lesgold 
Learning R&D Center 
University of Pittsburgh 
Pittsburgh, PA 15260 

Dr. John Levine 
Learning R&D Center 
University of Pittsburgh 
Pittsburgh, PA 15260 



Library 

Naval Training Systems 
Center 

Orlando, FL 32813 



Library 

Naval War College 
Newport, Rl 02940 



Science and Technology 
Division 

Library of Congress 
Washington, DC 20540 

Dr. Elaine Marsh 



ERIC 



4S 



Naval Center for Applied 
Research 

in Artificial Intelligence 
Naval Research Laboratory 
Code 5510 

Washington, DC 20375- 
5000 

Dr. Sandra P. Marshall 
Dept. of Psychology 
San Diego State University 
San Diego, CA 92182 



Dr. Joseph C. McLachlan 
C^de 52 

Navy Personnel R&D Center 
San Diego, CA 92152-6800 

Dr. James McMichael 
Technical Director 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 



Dr. Douglas L. Medin 
Department of Psychology 
University of Illinois 
603 E. Daniel Street 
Champaign. IL 61820 

Dr. James R. Miller 
MCC 

3500 W. Balcones Center 
Dr. 

Austin, TX 78759 



Dr. William Montague 

NPRDC Code 13 

San Diego, CA 92152-6800 



Dr. Randy Mumaw 
Training Research Division 
HumRRO 

1100 S. Washington 
Alexandria, VA 22314 



Dr. Allen Munro 
Behavioral Technology 
Laboratories - USC 



1845 S. Elena Ave., 4th 
Floor 

Redondo Beach, CA 90277 



Dr. Donald A. Norman 
C-015 

Institute for Cognitive 
Science 
University of California 
La Jolla, CA 92093 



Deputy Technical Director 
NPRDC Code 01 A 
San Diego, CA 92152- 
6800 

Director, Training 
Laboratory, 

NPRDC (Code 05) 
San Diego, CA 92152-6800 

Director, Manpov/er and 
Personnel 
Laboratory, 
NPRDC (Code 06) 
San Diego, CA 92152-6800 



Director, Human Factors 

& Organizational Systems 
Lab, 

NPRDC (Code 07) 
San Diego, CA 92152-6800 

Library, NPRDC 
Code P201L 

San Diego, CA 92152-6800 



Technical Director 

Navy Personnel R&D Center 

San Diego, CA 92152-6800 



Commanding Officer, 

Naval Research 
Laboratory 
Code 2627 

Washington, DC 20390 



Office of Naval Research, 

Code 1142 
800 N. Quincy St. 
Arlington, VA 22217-5000 



Office of Naval Research, 

Code 1142BI 
800 N. Quincy Street 
Arlington, VA 22217-5000 

Office of Naval Research, 

Code 1142CS 
800 N. Quincy Street 
Arlington, VA 22217-5000 
(6 Copies) 



Office of Naval Research, 

Code 1142PS 
800 N. Quincy Street 
Arlington, VA 22217-5000 

Psychologist 

Office of Naval Research 
Branch Office, London 
Box 39 

FPO New York, NY 09510 



Special Assistant for Marine 

Corps Matters, 
ONR Codo OOMC 
800 N. Quincy St. 
Arlington, VA 22217-5000 

Dr. Judith Orasanu 
Basic Research Office 
Army Research Institute 
5001 Eisenhower Avenue 
Alexandria, VA 22333 



Military Assistant for 
Training and 

Personnel "technology, 
OUSD (R & E) 
Room 3D129, The Pentagon 
Washington, DC 20301- 
3080 



Dr. Steven E. Poltrock 
MCC 



ERLC 



46 



3500 West Balcones Center 
Dr. 

Austin, TX 78759-6509 

Dr. Steve Reder 
Northwest Regional 

Educational Laboratory 
400 Lindsay BIdg. 
710 S.W. Second Ave. 
Portland. OR 97204 

Dr. J. Wesley Regian 
AFHRL'IDI 

Brooks AFB, TX 78235 

Dr. Gilbert Ricard 
Mail Stop K02-14 
Grumman Aircraft Systems 
Bethpage. NY 11787 

Dr. J. Jeffrey Richardson 
Center for Applied Al 
College of Business 
University of Colorado 
Boulder, CO 80309-0419 

Dr. Linda G. Roberts 
Science, Education, and 

Transportation Program 
Office of Technology 
N Assessment 
Congress of the United 
States 

Washington, DC 20510 

Dr. William B. Rouse 
Search Technology, Inc. 
4725 Peachtree Corners 
Circle 
Suite 200 

Norcross, GA 30092 

Dr. James F. Sanford 
Department of Psychology 
George Mason University 
4400 University Drive 
Fairfax, VA 22030 

Dr. Walter Schneider 
Learning R&D Center 
University of Pittsburgh 
3939 O'Hara Street 
Pittsburgh, PA 15260 



Dr. Janet W. Schofield 
816 LRDC Building 
University of Pittsburgh 
3939 O'Hara Street 
Pittsburgh, PA 15260 

Dr. Colleen M. Seifert 
Institute for Cognitive 
Science 

Mail Code C-015 
University of California, 
San Diego 
La Jolla, CA 92093 

Dr. Ben Shneiderman 
Dept. of Computer Science 
University of Maryland 
College Park, MD 20742 

Dr. Randall Shumaker 
Naval Research Laboratory 
Code 5510 

4555 Overlook Avenue, 
S.W. 

Washington, DC 20375- 
5000 

Robert L. Simpson, Jr. 
DARPA/ISTO 
1400 Wiison Blvd. 
Arlington, VA 22209- 
2308 

Dr. Richard E. Snow 
School of Education 
Stanford University 
Stanford, CA 94305 

Dr. Elliot Soloway 
Yale University 
Computer Science 
Department 
P.O. Box 2158 
New Haven, CT 06520 

Dr. Richard C. Sorensen 
Navy Personnel R&D Center 
San Diego, CA 92152-6800 



Dr. John Tangney 
AFOSR/NL, BIdg. 410 
Boiling AFB, DC 20332- 
6448 



Dr. Perry W. Thorndyke 
FMC Corporation 
Central Engineering Labs 
1205 Coleman Avenue, Box 
580 

Santa Clara, CA 95052 

Dr. Douglas Towne 
Behavioral Technology Labs 
University of Southern 
California 
1845 S. Elena Ave. 
Redondo Beach, CA 90277 

Headquarters, U. S. Marine 
Corps 

Code MPI-20 
Washington, DC 20380 

Dr. Jerry Vogt 

Navy Personnel R&D Center 

Code 51 

San Diego, CA 92152-6800 

Dr. Kent E. Williams 
Inst, for Simulation and 

Training 
University of Central 

Florida 

P.O. Box 25000 
Orlando, FL 32816-0544 

Dr. Martin F. Wiskoff 
Defense Manpower Data 
Center 

550 Camino El Estero 

Suite 200 
Monterey, CA 93943-3231 

Dr. Wallace Wulfeck, III 
Navy Personnel R&D Center 
Code 51 

San Diego, CA 92152-6800 



Dr. Joseph L. Young 
National Science Foundation 
Room 320 
1800 G Street, N.W. 
Washington, DC 20550 



47 



The University of Michigan, as an Equal Opportunity/ Affirmative 
action employer, complies with applicable federal and state laws 
prohibiting discrimination, including I'itle DC of the Education 
Amendments of 1972 and Section 504 of the Rehabilitation Act of 
1973. It is the policy of The University of Michigan that no person, 
on the basis of race, sex, color, religion, national origin or ancestrj', 
age, marital status, handicap, or Vietnam-era veteran status, shall 
be discriminated against in employment, educational programs 
and activities, or admissions. Inquiries or complaints may be 
addressed to the University's Director of Affirmative Action, Title 
DC and Section 504 Compliance, 2012 Fleming Administration 
Building, Ann Arb^r, Michigan 48109-1340. (313) 763-0235. 

The Regents of the University: 

Deane Baker Paul Brown Neal Nielsen 

Philip Power Thomas Roach Veronica Smith 

Nellie Vamer James Waters Robben Heming (ex officio) 



